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(54) Tide: METHOD FOR COLLECTING AND PROCESSING VISUAL AND SPATIAL POSITION INFORMATION 



(57) Abstract 

Visual (130) and spatial (120) position information is collected and 
processed (132) for the formation of a geographic information database. In 
addition, the information collected and processed can be used to update the 
United States Census Bureau TIGER/Line files (134) to be more accurate files 
(146). Information collected and processed can also be used to accurately 
determine the spatial position of an object (144) seen in the collected visual 
information. The method includes the high speed collection and correlation 
of video images and spatial position information. 
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METHOD FOR COLLECTING AND PROCESSING VISUAL AND 
SPATIAL POSITION INFORMATION 

BACKGROUND OF THE INVENTION 
The present invention relates to a method of using integrated 
5 video and spatial position information to maintain and expand information 
located in a geographic information system. The method includes the high 
speed collection of video and spatial position data as well as fast and 
efficient storage of the data. Such a method is of particular use in updating 
the United States Census Bureau TIGER/Line™ files to improve the 
10 accuracy of coordinates contained therein, add missing street and address 
information, and add vehicle routing data. The method may also be used 
to create files similar to TIGER/Line files in areas where such files do not 
already exist. 

The TIGER/Line files are a computer readable geographic 
15 database for the United States. The original sources of the database include 
U.S. geological survey maps, the 1980 U.S. Census Bureau DIME files, and 
various other sources. While these geographic files are usable as a base for 
a geographic information system, they are not recommended for detailed 
applications which require a high degree of positional accuracy. This is 
20 because the positional accuracy of the information contained in the file is 
limited by the scale of the U.S. Geological Survey maps from which the files 
are created. 

In addition, information in the TIGER/Line files is only as 
complete and accurate as the source documents used by the U.S. Census 
25 Bureau and errors often occur in the compilation and translation of 
information from source documents. Further, the information is only as 

9 

current as the year in which the source documents were compiled and 
translated. 
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Because the information contained in a geographic information 
system such as the TIGER/Line files must be continually updated to reflect 
the most recent changes in the information contained therein, it is desirable 
to provide a method for collecting and processing video and spatial position 
5 information in a manner that allows updated attribute information to be tied 
to previously obtained spatial positional information. 

One way in which to acquire accurate spatial position 
information is througn the use of the Global Positioning System (GPS). The 
Global Positioning System is a network of satellites which constantly 

10 transmits its location at known, precise times. Since the orbital position of 
each satellite is known, a GPS receiver on the surface of the earth can 
accurately measure the time it takes for the transmitted signal to reach the 
receiver's location on earth. This measurement provides the distance 
between the satellite and the receiver. To obtain three dimensional 

15 positioning, the GPS receiver takes this measurement from each of at least 
four orbiting satellites. Using geometric principles, the information gathered 
by the GPS receiver can be used to calculate the distance of each satellite 
with respect to the receiver and thus calculate an accurate spatial position 
on earth. 

20 While the Global Positioning System can provide accurate 

spatial position information, problems arise when the GPS receiver is not 
able to receive information from at least four orbiting satellites. This occurs 
when the GPS receiver is obstructed by geographic features such as canyons 
or bluffs as well as by man-made structures. Problems also arise when a 

25 signal transmitted by an orbiting satellite is reflected from a nearby object, 
giving the GPS receiver one or more incorrect readings from that satellite. 

A further problem with the use of the Global Positioning 
System to provide spatial position information is that spatial position 
readings are often provided by the GPS receiver at a relatively slow rate. 
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SUMMARY OF THE INVENTION 
The present invention provides a method for collecting and 
processing video and spatial position information for the formation of a 
geographic information database. The database can be further processed to 
5 create additional databases which can be used for a variety of application 
such as to provide access to video images of streets and their surroundings. 
In addition, the information collected and processed according to the 
method of the present invention can be used to update the United States 
Census Bureau TIGER/Line computer map files to improve the accuracy 
10 of the files and add information. 

The present invention includes a collection system comprising 
video cameras, a GPS receiver, an inertial navigation system (INS) and a 
control computer. The collection system is mounted in a moving vehicle 
such as a van. The GPS receiver provides satellite-based spatial position 
15 information and precise time values to the control computer while the INS 
provides kinematic-based spatial position information to the control 
computer. At the same time, a video tape recorder controlled by the control 
computer records interleaved video images from the video cameras onto a 
single video tape. 

20 Each frame of the video tape has a time code associated 

therewith. This time code is recorded by the control computer along with 
the current spatial position information provided by the GPS receiver and 
the INS at the time that frame is recorded. In this way, each recorded video 
image can be correlated with the spatial position of the camera at the time 

25 the image was recorded. 

A GPS base receiver is located at a known position and 
provides absolute spatial position information and absolute time values from 
the same GPS satellites as those used by the GPS receiver in the collection 
system. This position information is used to remove location errors in the 
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recorded GPS spatial position information by a process known as differential 
processing. 

A centerline offset process is used to determine the true 
centerline of each street from which data was collected. The centerlines are 
5 used to create street segments in a process called street segmentation and 
each street segment is associated with a sequence of video images. The 
relationship between each segment and associated video images is stored in 
a database. Additional attribute information databases can be created for 
address location, vehicle routing and point location applications among 
10 others. 

Because the method of the present invention records the 
spatial position at the time each video image is recorded, the video images 
can be later reviewed as many times as is necessary to validate the accuracy 
and completeness of the attributes shown therein. In addition, because the 
15 INS provides six-dimensional spatial position information, the present 
invention allows the location of a point to be accurately determined from 
any two video images containing the point. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a perspective view of a vehicle in which the 
20 collection system of the present invention can be located. 

Figure 2 is a schematic diagram showing the collection system 
of the present invention. 

Figure 3 is a schematic diagram showing a portion of the 
image collection system of the present invention. 
25 Figure 4 is a schematic diagram showing a portion of the 

image collection system of the present invention. 

Figure 5 shows the six-dimensional relative position of the 
vehicle provided by the inertial navigation system. 
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Figure 6 is a schematic diagram showing data inputs and 
outputs of the collection system of the present invention. 

Figure 7 shows a series of steps which are followed in the 
present invention. 

Figure 8 shows the manner in which a portion of a street 
centerline is determined. 

Figure 9 shows a street segment created according to the 
method of the present invention. 

Figure 10 shows a portion of a street segment created 
according to the method of the present invention. 

Figure 11 shows the relationship between a series of recorded 
video images and a street segment. 

Figure 12 is a schematic diagram showing a number of ways 
in which updated street segments can be used. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
In the method of present invention, accurate visual and 
positional information is collected in a rapid and efficient manner by a 
collection system 10. The information is then stored for post-collection 
processing, which places the data in a more easily usable form. After 
processing, various extraction processes are used to create a variety of 
geographic/informational databases. This is accomplished in part through 
the creation of street segments and the creation of indirect relationships 
between the segments and recorded video images. Generally, the collection 
system 10 is located on a moving platform such as a moving vehicle 11, as 
shown in Figure 1. 

The collection system 10 for collecting video and positional 
information is shown in Figure 2 and includes a land reference mapping 
computer 12, a data acquisition control computer 16, a video tape recorder 
20, an image control system 22 including a digital matrix router 23, a GPS 
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receiver 24 and an inertial navigation system (INS) 28. In addition, the 
collection system 10 includes a first computer monitor 30, a second computer 
monitor 32, a keyboard 36, a selecting switch 38 and a trackball 40. Each 
of these components are located in the moving vehicle 11. A plurality of 
5 batteries located in the vehicle 11 provide power to the collection system 
equipment even when the engine is not running. AC power is provided to 
the equipment through the use of a power inverter. The collection system 
10 further includes a GPS base receiver 42 which is located at a known, 
fixed location separate from the rest of the collection system 10. 

10 The control computer is linked with the video tape recorder 

20, the digital matrix router 23, the GPS receiver 24, the INS 28 and the 
mapping computer 12. The keyboard 36 is selectably linked to both the 
control computer 16 and the mapping computer 12 through the selecting 
switch 38. In addition, the trackball 40 is electrically connected to the 

15 mapping computer 12. The first computer monitor 30 is linked to the 
mapping computer 12 while the second computer monitor 32 is linked to the 
control computer 16. 

The data acquisition control computer 16 operates a set of 
nested asynchronous interrupt processing routines to provide real time 

20 control of all data acquisition equipment. The control computer 16 controls 
the signal routing of all data collected as well as the recording of the data 
by the video tape recorder 20. A video equipment controller 44 such as a 
Video Logic V-LAN™ controller is located in the control computer 16 and 
provides control of the video tape recorder 20 while it is recording. The 

25 video equipment controller 44 is linked with the video tape recorder 20 by 
a RS-422 serial communications link. 

Both the GPS receiver 24 and the INS 28 are linked to the 
control computer 16 by RS-232 communications links and provide spatial 
position information to the computer 16. The GPS receiver 24 automatically 
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acquires, tracks and selects the satellites to be used for position and time 
calculations and provides location readings to the control computer 16 at a 
rate of approximately three times per second. The INS 28 provid.es the 
control computer 16 with relative positional locations at the rate of thirty 
5 locations per second. The positional information or data received by the 
control computer 16 is transferred to a hard drive for permanent storage. 
Both the mapping computer 12 and the control computer 16 include a 
backup tape to provide for backup of the collected data; 

The land reference mapping computer 12 provides real time 

10 tracking of the location of the moving vehicle 11. This is accomplished 
through an RS-232 communications link between the mapping computer 12 
and the control computer 16, allowing positional information to be 
transmitted to the mapping computer 12. A street map is displayed on the 
first computer monitor 30 showing both the current and previous locations 

15 of the vehicle 11. The map is initially based on current U.S. Census Bureau 
TIGER/Line files. The mapping computer 12 thus acts as a navigation aid 
for routing the moving vehicle 1 1 during the data collection process. 

Figures 3 and 4 each show a portion of the image control 
system 22 of the present invention, which is not completely shown in a single 

20 figure for ease of illustration. Figure 3 shows the portion of the image 
control system 22 in which the video signals are routed. This portion of the 
image control system 22 comprises the digital matrix router 23, ten video 
cameras 50A, 50B, 50C, 50D, 50E, 50F, 50G, 50H, 501 and 50J (collectively 
referred to as 50), four video monitors 54A, 54B, 54C and 54D (collectively 

25 referred to as 54), a four-view generator 62, a time base corrector 64, a 
vectorscope 68 and ten video camera control units 70A, 70B, 70C, 70D, 70E, 
70F, 70G, 70H, 701 and 70J (collectively referred to as 70). Each of the 
video cameras 50 is mounted in the moving vehicle 11 at a known position 



WO 95/32483 



PCT/US95/06232 



-8- 

relative to the GPS receiver 24 and oriented to capture a desired view. The 
orientation of the video cameras 50 are as follows: 



CAMERA 


VIEW 


50A 


Driver's Front 


50B 


Boulevard Front 


50C 


Stereo Front 


50D 


Right Side - Short Focal Length 


50E 


Right Side - Long Focal Length 


50F 


Driver's Rear 


50G 


Boulevard Rear 


50H 


Stereo Rear 


501 


Left Side - Short Focal Length 


50J 


Left Side - Long Focal Length 



15 Video cameras SOB and 50G are located on remote control 

pan/tilt units and can be adjusted through the use of two joystick controls 
while cameras SOB, 50E, 50G and 501 have remote zoom and focus controls. 
Each video camera 50 is electrically connected to a corresponding camera 
control unit 70 which is used to provide normal camera head controls. Each 

20 camera control unit 70 is electrically connected to one of a plurality of 
inputs 74 of the digital matrix router 23 and provides a signal to the router 
23. The distance between the GPS receiver 24 and each video camera 50 
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is measured so that the spatial position of each camera 50 can be 
determined from the collected spatial position information. 

The digital matrix router 23 is linked to the data acquisition 
control computer 16 which controls the operation of the router 23. The 
5 router 23 provides at a plurality of router outputs 78 any desired 
combination of signals received at its inputs 74 as instructed by software 
located in the control computer 16. Four router outputs 78 are electrically 
connected to the four-view generator 62. Each of three router outputs 78 
are electrically connected to first, second and third video monitors 54A, 54B 
10 and 54C, respectively, through the vectorscope 68. The two remaining router 
outputs 78 are electrically connected to the video tape recorder 20 and a 
fourth video monitor 54D respectively. Both the time base corrector 64 and 
a first output 80 of the video tape recorder 20 are electrically connected to 
router inputs 74. 

15 The four-view generator 62 has four inputs 82, allowing signals 

from four of the video cameras 50 to be input simultaneously. The 
generator 62 reduces the image represented by each signal to one-fourth of 
its original size and then combines the reduced images to form a single 
video image by placing each of the reduced images into one of the four 

20 corners of an output image. This output image is referred to as a four-view 
image and is provided at an output 84 of the four-view generator 62, which 
is electrically connected to an input 86 of the time base corrector 64. The 
time base corrector 64 holds the four-view image in a buffer until the image 
is selected by the digital matrix router 23. This buffer is necessary because 

25 four-view images created by the four-view generator 62 require 
approximately one-tenth of a second to become sufficiently stable to be 
accurately recorded by the video tape recorder 20. 

In addition to sending signals to the four-view generator 62 
and to the video tape recorder 20, the digital matrix router 23 routes images 
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from the video cameras 50 and the time base corrector 64 to the four video 
monitors 54. The order and duration with which each of the images is 
routed to the monitors 54 is determined by software in the control computer 
16 which then instructs the router 23 which images to provide. The images 
5 provided by the router 23 to the first, second and third video monitors 54A, 
54B and 54C are routed through the vectorscope 68. The vectorscope 68 
monitors the signals from the video sources provided by the router 23 and 
provides those signals to the first, second and third video monitors 54A, 54B 
and 54C for normal video monitoring. The vectorscope 68 is used as a 
10 diagnostic source and provides diagnostic signals from two diagnostic outputs 
90 to channel B of both the first video monitor 54A and the second video 
monitor 54B. 

The video tape recorder 20 comprises a Sony PVW-2800 
BetaCam SP Video Tape Recorder and is controlled by the data acquisition 

15 control computer 16. A second output 94 of the video tape recorder 20 is 
electrically connected to a second input 96 of the fourth video monitor 54D 
and provides to channel B of the monitor 54D a signal containing the video 
images being recorded. The video tape recorder 20 records images provided 
from the digital matrix router 23 on Sony Betacam SP video tapes which are 

20 generally 90 minutes in length. Each tape includes three video component 
tracks, two audio tracks, and a Society of Motion Picture and Television 
Engineers (SMPTE) time code track. 

The portion of the image collection system 22 that controls the 
precise timing of all video equipment is shown in Figure 4. A standard 

25 video black reference timing generator 100 is used to synchronize the timing 
of the image control system 22. The timing generator 100 has a first 
generator output 102 which is electrically connected to the input of a first 
six-channel distribution amplifier 104 and a second generator output 106 
which is electrically connected to the input of a second six-channel 
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distribution amplifier 108. Both the first amplifier 104 and the second 
amplifier 108 amplify a black reference timing signal generated by the timing 
generator 100 and provide the timing signal at each of six first and second 
amplifier outputs 110 and 112, respectively. 
5 The first distribution amplifier 104 provides the timing signal 

to the video tape recorder 20, the digital matrix router 23, the four view 
generator 62, the vectorscope 68 and the time base corrector 64. The 
second distribution amplifier 108 provides the timing signal to the video 
equipment controller 44, the second camera control unit 70B, the fourth 

10 camera control unit 70D, the sixth camera control unit 70F, the eighth 
camera control unit 70H and the tenth camera control unit 70J. The first, 
third, fifth, seventh and ninth camera control units 70A, 70C, 70E, 70G and 
701 are linked to the second, fourth, sixth, eighth and tenth camera control 
units 70B, 70D, 70F, 70H and 70J, respectively, to allow the timing signal to 

15 be provided to the remaining camera control units. Electrical circuitry with 
each video device synchronizes a National Television Standards Committee 
(NTSC) video blanking interval with the black reference timing signal. 

Before recording on the video tape, the cable connections to 
the video tape recorder 20 are modified to provide a video black reference 

20 signal to the video tape recorder 20, which creates a series of individual 
frames on a blank tape. A time code generator built into the video tape 
recorder 20 provides an accurate marking of each of these video frames with 
a SMPTE time code. The SMPTE time codes are written on the videotape 
in a continuous stream of non-drop frame time codes in an 

25 hours: mi nutes:seconds:frame format which comprises thirty frames (0-29) per 
second, 60 seconds per minute and 60 minutes per hour. The marking of 
the video tape permits video images to be accurately recorded to the video 
frames during the data collection process. 
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The control computer 16 can send a set of instructions to the 
digital matrix router 23 to perform a desired continuous sequence of 
predetermined routings called a cycle. Instructions sent to the router 23 are 
timed by the video equipment controller 44. Instructions are sent by the 
5 control computer 16 to the router 23 before the NTSC video blanking 
interval of the desired video frame SMPTE time code and router switching 
commands are queued with the router 23. When the desired NTSC video 
blanking interval of the black reference signal is detected by the router 23, 
the router inputs 74 are connected to the router outputs 78 as specified by 

10 the commands from the control computer 16. The connections between the 
router inputs and outputs 74 and 78 are completed before the end of the 
NTSC video blanking interval to provide seamless switching as often as once 
per blanking interval. 

Each of the video cameras 50 is capable of obtaining images 

15 at a rate of 30 frames per second. In addition, the digital matrix router 23 
is capable of altering the routing of the signals received its inputs 74 thirty 
times a second. The video images provided at the router outputs 78 can 
thus be altered as frequently as thirty times per second. 

Within the instructions sent to the router 23 is contained a 

20 sequence of the video cameras 50 from which video images are to be 
recorded by the videotape recorder 20. In this way, images from each of the 
video cameras 50 can be recorded in a desired order by a single video 
recording device. A typical 18-frame recording cycle is given as follows: 
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FRAME 


CAMERA VIEW 


FRAME 


CAMERA 
VIEW 


1 


1 


10 


4-View 

1 - 8 

2 - 7 


2 


2 


11 


8 


3 


4 


12 


4 


4 


4 


13 


9 


5 


9 


14 


4-View 
1 - 8 
10 - 4 


6 


4-View 
1 - 3 
6 - 8 


15 


7 


7 


10 


16 


10 


8 


10 


17 


5 


9 


5 


18 


4-View 
9-5 
10 - 4 



The given recording cycle is repeated continuously during the 
data collection process, resulting in the recording of a plurality of interleaved 
video images from multiple video image sources onto a recording medium. 
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As can be seen, not only can the view from any video camera 50 be 
recorded, but so can any desired four-view image. The occasional recording 
of the same camera view for two consecutive frames provides sequential side 
views which can be useful in terrestrial photogrammetry. 
5 The recording cycle shown above can be dynamically 

configured during recording. The computer keyboard 36 permits up to ten 
preset recording cycles. When a new recording cycle is selected, the control 
computer 16 begins the new cycle immediately after completion of the then 
current recording cycle. The precise timing described above permits a 

10 change in the recording cycle, as well as a change in the signals provided to 
the video monitors 54, to occur within a single NTSC blanking interval. 

During the data collection process, which is organized into a 
series of "runs", the video cameras 50 continually send image signals to the 
digital matrix router 23 through the camera control units 70. Pursuant to 

15 instructions from the control computer 16, the router 23 provides to the 
video tape recorder 20 the signal from a selected one of the image sources, 
which include the ten video cameras 50 and the four-view generator 62. The 
images provided to the video tape recorder 20 are recorded at the rate of 
thirty per second in a predetermined order comprising a cycle. In addition, 

20 audio comments from both the driver and navigator of the moving vehicle 
1 1 are recorded by the video tape recorder 20. 

While the video tape recorder 20 is recording video images, 
the GPS receiver 24 acquires thiee dimensional fixed location data from the 
NASA NAVSTAR Satellite Network at selected time intervals and at 

25 selected locations. This fixed location data is provided to the control 
computer 16 along with the Greenwich Mean Time (GMT) transmitted by 
the satellites. The control computer records the location data from each 
positional reading provided by the GPS receiver 24, the data being given in 
terms of latitude, longitude and elevation. The video equipment controller 
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44 provides the control computer 16 with the SMPTE time code of the 
frame on which the video tape recorder 20 is recording for each positional 
reading of the GPS receiver that it receives and records. The SMPTE time 
code is then recorded by the control computer 16 with the corresponding set 
5 of GPS location data. 

The inertial navigation system 28 comprises a set of rotation 
and acceleration sensors and continually determines the relative position of 
the vehicle 11 in terms of latitude, longitude, elevation, pitch, roll and yaw 
as shown in Figure 5. The INS 28 can thus provides spatial position with six 

10 degrees of freedom relative to previous positional readings taken by the GPS 
receiver 24. The INS provides this six-dimensional relative position data at 
the rate of thirty times per second and the data is recorded by the control 
computer 16. Preferably, the INS 28 utilizes high quality laser gyroscopes 
which provide extremely accurate rotational/attitude data. The INS 28 can 

15 also include an odometer coupled to an undriven wheel of the vehicle 11, 
which reduces the need to stop for re-alignment of the INS 28. 

The video equipment controller 44 provides the control 
computer 16 with the SMPTE time code of the frame on which the video 
tape recorder 20 is recording for each positional reading of the INS 28 that 

20 it receives and records. The SMPTE time code is then recorded by the 
control computer 16 with the corresponding set of INS position data. The 
recording of GPS and INS spatial position data along with the corresponding 
SMPTE time code allows the position data to be interpolated by SMPTE 
time code during post-collection processing. 

25 By using the inertial navigation system 28, the collection 

system 10 of the present invention is able to collect accurate positional 
readings under many circumstances in which the GPS receiver 24 is unable 
to obtain accurate position data. This often occurs when the GPS receiver 
cannot obtain enough satellite signals to calculate a position and when the 
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signals from one or more of the satellites is reflected off of a building or 
other surface. 

If the vehicle must be stopped for an extended period of time, 
or if a previously recorded street must be driven again to reach a desired 
5 location, a pause command is given by the navigator to stop the recording 
of both video images and positional information. When it is desired to begin 
recording data, a resume command is given. The control computer 16 
records the SMPTE time code of the last frame on which the video tape 
recorder 20 has recorded after the issuance of each pause command and the 

10 first frame on which recording begins after each resume command. 

The flow of data in the collection system during a data 
collection run is shown in Figure 6. The collected data includes a video tape 
114, GPS and INS spatial position data 115 and a record of each pause and 
resume command given 116. The video tape 114 contains all of the 

15 recorded video images as well as all audio information 118 from the driver 
and navigator. The video tape 114 is converted to a digital compressed 
format and stored in a digital video file for easy random access during post- 
collection processing. A correlation table is created to maintain the 
relationship of each digital video frame SMPTE time code with the SMPTE 

20 time code on the original video tape. The GPS and INS position data 115 
provided from the GPS receiver 24 and the INS 28 is recorded to a digital 
file. The recorded sbc-dimensional spatial position data is indexed by 
SMPTE time code, allowing it to be correlated with each recorded video 
image or frame. Each set of position data also includes the corresponding 

25 GMT, which identifies when the information was collected and allows the 
SMPTE time codes to be indexed by GMT. The record of each pause and 
resume command is indexed by SMPTE time code. 

During each run, the GPS base receiver 42, which is not shown 
in Figure 6, collects three-dimensional position data from the NASA 



WO 95/32483 



PCT/US95/06232 



-17- 

NAVSTAR Satellite Network at the same intervals as the GPS receiver 24 
in the vehicle 1 1. Each set of position data is recorded in a base station file 
along with a corresponding GMT provided by the GPS satellites. The 
collection of position data by the GPS base receiver 42 must begin prior to 
5 the beginning of each run and must be continuous throughout each run. In 
addition, the GPS base receiver 42 must be positioned so that there are no 
obstructions between the receiver 42 and the GPS satellites. 

Figure 7 shows a sequence of steps that are performed during 
the method of the present invention. As described above, both visual and 

10 spatial position data is collected by the data collection system 10, as shown 
by box 120, and spatial position data is collected using the GPS base receiver 
42, as shown by box 122. The spatial position data from the GPS receiver 
24 and the GPS base receiver 42 is then differentially post-processed as 
shown by box 124. The differentially processed GPS data and the INS 

15 spatial position data is correlated as shown by box 126, errors in the INS 
position data are corrected, and the GPS and INS position data is combined 
as shown by box 128. 

Next, the off>et between the spatial position of the vehicle 11 
and the street centerline is calculated as shown by box 132 using the images 

20 on the collected video tape 130. Street centerline points are then plotted on 
a computer-aided design display system as shown by box 136. Street 
segment nodes from a TIGER/Line file 134 are updated as shown by box 
138 and the updated street segment nodes allow new street segments to be 
created as shown by box 140. Shape points are then calculated for the street 

25 segments as shown by box 142 and a video image-segment relationship is 
established as shown by box 144. This provides a complete data file 146 
containing accurate, updated street segments having a known relationship 
with the recorded video images. Each of these steps is more completely 
described below. 
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The United States Department of Defense intentionally 
introduces errors into the NAVSTAR satellite information provided by each 
satellite, which results in position data inaccuracies. This error is removed 
prior to processing the collected information by differential processing of the 
5 position data acquired by the known position GPS base receiver 42 with that 
of the vehicle-based GPS receiver 24. The position data recorded by the 
GPS base receiver 42 is indexed by GMT time code and is correlated with 
the position data recorded by the GPS receiver 24 through the use of the 
GMT time codes. Differential processing yields a corrected three- 

10 dimensional spatial position of the GPS receiver 24 in terms of latitude, 
longitude and elevation for each position at which a GPS reading was taken. 

Once an absolute position for each GPS reading is determined, 
the recorded relative position data from the INS 28 can be converted into 
absolute spatial position data in terms of latitude, longitude and elevation. 

15 These absolute positions are then combined with the recorded pitch, roll and 
yaw of the vehicle to form a completed data file of accurate six-dimensional 
spatial positions which can be referenced by date, SMPTE time code, GMT 
and run. During this processing, the INS position data is corrected when 
discrepancies exist between the position data provided by the INS 28 and 

20 that provided by the GPS receiver 24. When this occurs, the position data 
provided by the INS 28 is corrected by prorating the error found in speed 
and direction at the discrepant point back through the records to the 
previous GPS positional reading. 

After the data collection and post-processing is completed, the 

25 information is ready for visual data correlation and extraction. This 
extraction process is performed entirely after both the data collection 
process and the data post-processing. By separating the collection, post- 
processing and extraction processes, a greater amount of data can be 
captured and a greater amount of information extracted than if they were 
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performed simultaneously. The information on which the extraction process 
is performed includes the video images recorded by the video tape recorder 
20, the six-dimension, absolute spatial positions of the vehicle 11, and the 
record of pause and resume commands. 
5 A file containing the absolute spatial positions of the vehicle 

11 along with the corresponding SMPTE time codes is loaded into a 
computer-aided design (CAD) display system which is not shown. The 
display system plots the X and Y coordinates of each spatial position 
recorded in the file as a point. The spacing between the points varies based' 

10 on the velocity of the vehicle 1 1 and the visual trail of plotted points defines 
the true path of the vehicle 1 1. Generally, the vehicle 1 1 travels to one side 
of the center of the street. Further, the spatial position data is collected 
relative to a single point within the vehicle 11. Therefore, the collected 
spatial position data does not relate directly to the true center of the street 

15 and the offset of the vehicle 11 from the actual center of the street must be 
calculated to determine the true center of the street. 

To determine the centerline of the street, an operator 
combines the display of plotted points representing the true spatial position 
of the vehicle 1 1 with a display of either the front or rear facing video 

20 images. The front or rear facing video image that corresponds to each 
plotted point can be recalled through a random access procedure through 
the use of the known SMPTE time code for each point. The operator 
selects the first plotted point in the segment and displays the corresponding 
video image in a digital video computer display window. The two views are 

25 calibrated to place the center of the video image directly in line with the line 
of travel of the vehicle 1 1. A line representing an imaginary projection from 
the video image source is displayed on the video display as a graphical 
cursor. This cursor can be moved left or right by a control interface such as 
a computer mouse or trackball. The distance between the cursor and the 
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graphical image vertical center is measured in terms of the number of pixels. 
The distance in pixels is translated to a distance in feet, using known 
information about the video source mounting and focal length. This distance 
in feet is the distance that the true vehicle position is offset from the center 
5 of the street and can be used to calculate the spatial position of the center 
of the street that corresponds to the vehicle position. 

The centerline determination process is shown in more detail 
in Figure 8. A video image 150 taken from a camera lens 152 of one of the 
cameras 50 is shown as being suspended above a street 154 on which t^ie 
10 centerline must be determined for purposes of illustration. A known vector 
D extends perpendicularly from the camera lens 152 to the surface of the 
street 154 and is defined in terms of a camera coordinate system that has its 
origin at the camera lens 152. Using a pointing device, a pixel 156 on the 
centerline in the video image 150 is chosen by an operator and a vector U 
15 is defined in coordinates from the camera coordinate system as extending 
from the camera lens 152 to the pixel 156. A multiplier L is then found for 
the vector U such that L*U extends to a point 158 in the plane of the 
surface of the street 154. The multiplier L is determined using the following 
equation: 

20 



Once the multiplier L is found, the vector L*U is determined. 
25 The vector L*U, which is defined in camera coordinates, is then converted 
to a global spatial position. This is done through a rotation of L*U to 
account for the rotational differences of roll, pitch and yaw between the 
camera coordinate system and the global spatial position and a translation 
of L*U to account for the offset differences of latitude, longitude and 
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elevation between the camera coordinate system and the global spatial 
position. The converted vector is then used to determine the spatial position 
of the point 158 on the centerline. 

When there is no marked centerline in the street, a pointing 
5 device with two additional markers can be used. The additional markers are 
equally spaced on either side of the centerline marker as equal distances 
forward of the vehicle 11. The width of the space between the two markers 
can be controlled to allow an operator to track the edges of the street. A 
computer automatically determines the point on the video image 150 of the 

10 street to which the vector U from the camera lens 152 should extend. 

When performed for each plotted point in a street segment, 
the centerline determination process generates a set of latitude, longitude 
and elevation coordinates representing the true street centerline. Once this 
process has been repeated for each point, a street centerline for the segment 

15 can be displayed on the CAD system. This process further takes into 
account changes in the location of the vehicle 11 relative to the street 
centerline as the vehicle 11 travels down the street, including those caused 
by traffic and obstructions. This centerline determination or offset 
calculation process is performed until a street centerline location has been 

20 calculated for each collected spatial position of the vehicle 11. 

A street segment is a set of ordered spatial positions, each of 
which is connected to an adjacent position, representing an approximation 
of a street centerline. Generally, street segments are defined in terms of two 
such spatial positions, known as a begin node and an end node, which 

25 represent the ends of the segment. To help further define curved street 
segments, a set of intermediate positions known as shape points are usually 
chosen between the begin and end nodes. Street segments that are straight 
generally do not require shape points. Street segments are initially defined 
from U.S. Census Bureau two dimensional TIGER/Line files and each 
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segment is uniquely identified based on a TIGER/Line file identification 
code called a TIGER/Line Identifier (TLID). 

Figures 9 and 10 illustrate the process of updating a street 
segment. Referring to Figure 9, a plurality of plotted points 190 is shown 
5 representing a set of spatial positions collected on a single street segment. 
A plurality of centerline position points 194, each of which is calculated from 
one of the plotted points 190 using the centerline determination process 
described above, comprise a centerline 198. A TIGER/Line street segment 
204 having a begin node 208 and an end node 210 is also shown in Figure 
10 9, along with an updated street segment 212 having a new begin node 216, 
a new end node 220 and a plurality of shape points 224. 

After the completion of the centerline determination process, 
the street segment approximating the calculated centerline position points 
194 is retrieved from a U.S. Census Bureau TIGER/Line file and is 
15 displayed on the CAD on top of the calculated street centerline 198. 
Because the TIGER/Line street segments are generally not very accurate 
approximations of newly calculated street centerlines, the TIGER/Line 
street segment 204 must be updated. This is done by relocating the begin 
node 208 and the end node 210 of the TIGER/Line street segment 204 and 
20 defining the shape points 224 to be used in the updated street segment 212. 

Figure 10 shows the relocation of the TIGER/Line segment 
begin node 208. Two points 228 and 232 from the centerline 198 define a 
first vector which is mathematically projected toward a second vector defined 
by points 236 and 240 from an intersecting street centerline 244. The 
25 mathematical intersection of these two vectors is determined and defines the 
latitude and longitude of the new begin node 216 for the updated street 
segment 212. An elevation value of the new begin node 216 is interpolated 
from a vertical intersection of the two vectors. The end node 210 of the 
TIGER/Line street segment 204 is relocated in the same manner as the 
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begin node 208, thus defining the new end node 220 for the updated street 
segment 212 in terms of latitude, longitude and elevation. After the begin 
and end nodes 208 and 210 of the TIGER/Line segment 204 have been 
updated, the node location attributes in the TIGER/Line file are updated. 
5 Because each node acts as both the begin node and end node for several 
street segments, a number of segments are updated with the relocation of 
each node. 

Refening to Figure 9, once the begin and end nodes 208 and 
210 of the TIGER/Line street segment 204 have been relocated, the 

10 centerline position points 194 between the new begin node 216 and the new 
end node 220 are statistically evaluated based on a deflection and distance 
formula to select the shape points 224 that will be used to graphically 
represent the updated street segment 212. The use of a few shape points 
224 rather than the large number of centerline position points 194 facilitates 

15 the efficient storage of the graphical database. A graphical display of the 
redefined street segment 212 having five shape points 224 is shown in Figure 
9. 

After the segment updating process has been completed, the 
segment database record is updated with reference to the new shape points 

20 224 by means of the standard TLID. This provides backward compatibility 
with all other TIGER/Line information. This process is repeated for all 
segments that were driven in the run. In addition to updating the segment 
location information, the run identifier and SMPTE time code for the new 
begin and end nodes 216 and 220 of the updated segment 212 is added to 

25 the segment database. This additional information provides the reference 
necessary to provide access to the digital video file. Through direct random 
access to the video file, a new digital video file is created for each view 
captured in a street segment. The run identifier and beginning SMPTE time 
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code provide a way to uniquely identify each group of recorded video images 
for each street segment. 

Once the physical street segment topology is defined, the video 
image sequences are associated with each street segment. Each video 
5 sequence is defined by a vehicle run identification number as well as a 
beginning and ending video frame SMPTE time code. By identifying each 
video sequence in this manner, a unique set of video images can be recalled 
for each segment. A video source sequence number is also stored to further 
define individual video sources within each interleaved video segment. The 
10 video source sequence number is the position in the video recording cycle 
of the beginning video image for the segment. This information, along with 
the video recording sequence noted in the run database, allows the video 
source to be determined for any SMPTE time code within the segment video 
range. 

15 A video image to segment matching process is used to match 

each street segment with at least one video image. Video image position 
information is used to find the video images which are geographically closest 
to both the begin node and the end node of a segment. First, a street 
segment having a known identifier is selected by an operator. The operator 

20 then selects a centerline position point within the segment. Using the 
SMPTE time code of the selected centerline position point, and working 
sequentially in both directions, each centerline position point at which a 
video image was recorded is compared to the begin node and the end node 
until the closest geographical match is found for each node. The SMPTE 

25 time codes for these two video frames are recorded as part of the video- 
segment relationship. 

The lower of the two recorded time codes is compared to the 
pause and resume records from the original vehicle run to determine the 
subset of video frames in which the segment was recorded. From the start 
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of this subset, the known recording cycle is used to calculate the video 
source sequence number for the segment. If the lower of the two recorded 
time codes is associated with the begin node, then the video images 
associated with the segment were taken in the direction of the street 
segment. If the lower of the two recorded time codes is associated with the 
end node, then the video images associated with the segment were taken in 
a direction opposite to the direction of the street segment. The video- 
segment direction relationship is recorded as part of the video-segment 
relationship. 

After the video-segment relationship is formed, the video 
sequences and street segments can be used for a variety of applications. In 
one such application, random digital video access techniques are used to 
reassemble the video sequence for a segment into a plurality of smaller 
video image sets containing sequential video images from a single video 
source. Each such image set is identified by run. video source and beginning 
SMPTE time code. A cross reference table is constructed to reference each 
image set with its associated street segment identifier. By reducing the large 
set of recorded digital video images into smaller sequences that are 
associated with street segments, maximum flexibility in data storage is 
provided. Through this street segment database, large amounts of digital 
video image sets can be manipulated and using current indexing 
methodology, up to 10 M street segments can be managed. 

The integration of video images with a geographic information 
system (GIS) requires multiple access methods for effective use in a GIS 
environment. To meet this requirement, both a set of segment-video and 
video-segment relationships is established and used to form a database. In 
addition, a number of secondary relationships are established from these 
video-segment and segment-video relationships to permit a wide variety of 
GIS applications. 
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Figure 11 helps illustrate the creation of a video-segment 
relationship and shows a street segment 250 having a begin node 252 and an 
end node 254. A first intermediate shape point 256 and a second 
intermediate shape point 258 are shown and are used to help define the 
5 segment 250 and are directly related to the begin and end nodes 252 and 
254. A series of video images are shown as points V1-V7 and each has a 
known position defined in terms of latitude, longitude, elevation, roll, pitch 
and yaw. Video image VI is directly related to the end node 254 and video 
image V7 is directly related to the begin node 252 through the matching 
10 process described above. The remaining video images are indirectly related 
to the begin node 252, the end node 254 and the shape points 256 and 258. 

The indirect relationship between the remaining video images 
and the shape points 256 and 258 is shown in the following example. The 
video image V3 can be defined in terms of the segment 250 as the 
15 perpendicular intersection between the end node 254 and the second shape 
point 258. This point of intersection can be described in absolute latitude, 
longitude and elevation or it may be expressed relative to the end node 254 
as the negative distance from the point of intersection to the end node 254. 
The relative position of video image V3 may also be expressed as the 
20 positive distance from the begin node 252 to the point of intersection which 
is equal to the distance from the begin node 252 to the first shape point 256 
plus the distance from the first shape point 256 to the second shape point 
258 plus the distance from the second shape point 258 to the point of 
intersection. 

25 Because the relationship between the video points V2-V6 and 

the begin node 252, end node 254 and shape points 256 and 258 is an 
independent one, a new relationship can easily be established if subsequent 
video and positional data were collected for the same segment. The new 
relationship may be established between any new video points and the 
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existing shape points 256 and 258 by establishing a new video to begin node 
252 and video to end node 254 relationship and calculating the point of 
intersection as described above. In other words, once the street segment, 
including the begin node, end node and shape points, has been established, 
5 it does not need to be updated and subsequent video images can be related 
to the segment in the same manner as were video images V1-V7. 

By creating a direct relationship between each node and the 
closest video image, a segment-video relationship is also created. As a result 
of this relationship, an ordered set of video images are associated with any 

10 given segment. This allows easy access to a set of video images and related 
attribute information for any given segment. Therefore, a segment position 
point described as a position relative to a begin node or an end node can be 
expressed as an absolute position in terms of video image positions. It is 
therefore possible to determine the closest video image to any given segment 

15 position, providing a user with the ability to examine a visual representation 
as well as related attribute information of a given position. As with the 
video-segment relationship, the video image positions are independent from 
the segment nodes and intermediate shape points. Therefore, the segment 
positions can be correlated in the same manner with video and positional 

20 information collected during subsequent runs. 

A number of applications in which the updated street segment 
and video-segment relationship data 146 can be used are shown in Figure 12. 
The applications include the creation and update of address ranges 282 and 
the integration of address attribute information 284, the creation and 

25 maintenance of street network topologies 286, the collection of vehicle 
routing information 288, the creation and maintenance of boundary polygon 
topologies and attributes 290 and 292, the accurate location of point features 
and their attributes 294 and 296, and the integration of street segment 



WO 95/32483 PCT/US95/06232 



-28- 

attributes with other GIS systems 298. These applications are discussed in 
more detail below. 

Attributes shown in a video image have an extended indirect 
relationship with segment positions. For example, a street address has an 
5 indirect address-segment relationship. By taking this relationship in 
combination with the indirect segment-video relationship, an address-video 
relationship can also be established. With such a relationship, an address 
seen in a video image can be compared to a calculated street address. If the 
two addresses are not the same, the visually observed address can be used 
10 to correct the model by which the address was calculated. 

Another example of the use of an extended relationship is in 
vehicle routing where the efficient collection of vehicle routing information 
is greatly enhanced by the video-segment data relationships. Vehicle routing 
information is divided into the two categories of node attribute information, 
15 which defines the relationship between two segments through a node, and 
segment attribute information, which defines the characteristics of the 
segments. By coupling these two sets of data with GIS based routing 
software, accurate vehicle navigation or routing can be accomplished. 

Node attribute information is collected through a visual 
20 examination of the recorded video images for each intersection. Each 
passage through an intersection is stored as a movement from a specified 
segment, through a specified node, to another specified segment. Using 
segment-video data relationships, an operator recalls the video images from 
any location around a node. The recalled video images are then visually 
25 inspected to determine the existence of a segment to segment connection as 
well as any traffic restrictions or controls. An indexed node attribute 
database is created to store all of this information for later reference. 

In the collection of segment attribute information, the video 
images for each segment are visually reviewed and evaluated for speed 
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limits t street type classification, one way status, weight or height restrictions, 
and other traffic controls. Each of these attributes is stored in a segment 
attribute database as a begin/end range with a left or right orientation. For 
example, a short no parking zone could be described as an attribute from a 
5 relative location X on the segment to the end of the segment and on the 
appropriate side of the street. 

Extended relationships can also be used to determine relative 
point positions which are points that are referenced to an attribute which is 
related to a segment. To determine a relative point position, a set of 
10 attributes must already exist and be linked to a segment by an attribute- 
segment relationship. The attribute to which the point is referenced can be 
visually verified by viewing the closest video image. Another type of point 
position is direct video position, which is the direct association of attribute 
data with the video frame identifier of the video image on which the 
15 attribute is found. 

The absolute position of a point can be found through the use 
of terrestrial photc^rammetry. In photogrammetry, multiple images 
containing a single chosen reference point are used to triangulate the 
position of the point relative to the cameras taking the images. This relative 
20 position is then used in combination with the known position of the cameras 
when the image was recorded to determine the absolute position of the 
chosen point. Once the absolute position of the point is determined, related 
attribute information can be visually determined and recorded. To help 
ensure the accuracy of the photogrammetric process, camera characteristics 
25 such as lens calibration information as well as specific camera mounting and 
orientation information must be taken into account. 

The present invention allows the photogrammetric process to 
be performed using any two images containing the chosen reference point. 
This is possible because the six-dimensional spatial position of each camera 
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is known and recorded for each video image taken. Therefore, the distance 
between where the two images were taken is known along with the 
orientations of the cameras from which the images were taken. The present 
invention thus allows the position of a chosen reference point to be 
5 determined through the use of images taken from cameras that are not in 
the same plane as well as through the use of multiple images from one 
camera. The performance of the photogrammetry process using images from 
a variety of cameras is aided through the use of interleaved image storage 
on the video tape. Further, the performance of the photogrammetry process 

10 using images from a variety of cameras allows positions to be calculated 
after the data collection process is completed. 

The ability to perform the photogrammetric process using 
images from any two cameras is a significant improvement over the 
traditional stereo photogrammetric approach. Such a traditional approach 

15 must compensate for a relatively short baseline between two cameras by 
using high resolution digital cameras, which requires extraordinarily large 
amounts of data to be stored. Further, when computing the position of a 
point or an object which is a great distance from the cameras, as must often 
be done in the stereo photogrammetric approach, the potential for error in 

20 calculating the position of the object is greatly increased. The present 
system, by allowing the use of any two images in which an object is found, 
allows the position of the object to be computed using images in which the 
object is relatively close to the camera. 

With the present invention, an accurate address database can 

25 be created and an existing database such as a TIGER/Line file can be 
updated. In addition, a database can be created using information collected 
and processed by the method of the present invention which allows the 
retrieval of video images associated with a street address. 
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All segments created through the process described above are 
referenced by a TIGER/Line identification number. This referencing 
method provides a unique identifier for all street segments in the United 
States. Further, new streets can be added to the database in compliance 
5 with the TIGER/Line identification system. Thus, street segments created 
through the update process described above can be directly related with a 
TLID. This relationship allows references to be made to all the data 
elements contained in the TIGER/Line file, allowing for total compatibility 
of the database provided by the present invention with this national 
10 standard. 

Each TLID contains an overall address range for the street 
segment it represents which is known as a nominal address range. In 
addition, each TLID supports both a left address range and a right address 
range as well as multiple sub-ranges. Each TLID is referenced to the 

15 coordinates of a node at each end of the street segment it represents. The 
coordinates of the node are used as a beginning point for the address range 
within the street segment. Therefore, the beginning address in each nominal 
address range is assumed to be located in the middle of a street intersection. 
For any specific street address, only one possible position may exist in the 

20 database. 

Address parsing is possible using the database provided by the 
present invention. Address parsing converts a user supplied address to a 
standard address within the database by matching it to a real address range 
in the database. This address range is referenced to a segment TLID, a 
25 relative TLID offset from one of the segment nodes and a side of the street. 
The side of the street on which an address is located is known as the parity 
of the address. 

Address ranges are stored internally as an offset from the 
beginning of each street segment. For instance, the beginning segment node 
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is considered to have a position offset of zero while the ending segment 
node is referred to as having an offset of one. The coordinate position of 
an address is based on the offset which is determined by taking the total 
segment length as defined by the shape points and prorating the distance 
5 along the segment. For each address, there will therefore be a pair of 
coordinates and a parity for the true address position. 

A video image can be recalled for any coordinate pair through 
the use of a coordinate-video image matching system. The video 
information is independent from the spatial position information and video 
10 frame calculations rely on the segment-video relationship discussed above. 
This process allows the retrieval of the nearest video image to a coordinate 
pair. 

An address range can be constructed after the street segment 
update process. The nominal address ranges are acquired from the 

15 TIGER/Line file for each street segment. The video for a given segment 
is displayed and a determination is made whether to insert an additional 
address range. Address matching is accomplished through the use of the 
information found in the TIGER/Line files prior to the address range 
construction. This information includes the identifiers for both the begin 

20 node and the end node of each segment, which contain the actual 
coordinates of the end points of the segment. Any shape points that are 
connected to the begin and end nodes to define the segment would also be 
accessed. Based on the node and shape points, the length of the segment 
is calculated. The information found in the TIGER/Line files also includes 

25 the beginning and ending address ranges that define the nominal address 
range for each segment. 

The address range construction process begins by viewing the 
video for a segment. As an address is observed, the video frame on which 
it is observed is converted to image coordinates. The actual offset from the 
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begin node is then determined through the video-segment relationship. The 
operator indicates whether the address is an exception to the anticipated 
range value. If the address is not an exception, the parity of the address is 
automatically recorded based on the camera view being observed. The raw 
5 information is then tabulated and includes for each address; the offset from 
the begin node, whether the address is an anticipated range value or an 
exception, the predicted parity, and the actual parity. 

A nominal range within a segment may be split by inserting 
additional range or exception points. This most frequently occurs when new 

10 streets are added. If the new street is perpendicular to an existing street, 
then the existing street is split and two new TLIDs are created for the split 
street using the TLID offset generated by the new node created by 
intersection with the new street. The operator will then set new nominal 
address ranges for the two new street segments and the old segment address 

15 ranges will be reassigned to the proper new segment TLID. A TLID for the 
new street will be created and address ranges will be processed similar to 
the initial TLID processing. 

While the collection system 10 was described as having a single 
GPS receiver 24, the present invention may be operated with two or more 

20 GPS receivers or with a GPS receiver having multiple antennas. Such a 
configuration might be desirable to provide a more frequent update of the 
vehicle position if a lower quality gyroscope is used in the INS 28. The 
update could then be used to more frequently correct any errors in the 
readings provided by the INS 28. Further, the collection system 10 can 

25 utilize a GPS receiver that is capable of providing its spatial position with 
six degrees of freedom. Spatial position information can also be provided 
by other devices such as odometers and pressure gauges and can be 
correlated with the spatial position data provided by the GPS receiver 24 
and the INS 28. 
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The position of each camera was described as being known 
through the: use of the recorded spatial positions of the vehicle 11. 
However, it is also possible to determine the position of a camera by using 
any three images of the same object captured at GPS coordinates not in a 
5 straight line. The information provided by three such images would be 
sufficient to determine the six-dimensional position of the camera, thus 
allowing photogrammetric analysis to be performed. 

While the collection system 10 was described as having only 
one video tape recorder 20, the system 10 can be configured to operate two 
10 or more video tape recorders. Further, the collection system 10 can be 
located on or in any moving platform, including a platform capable of being 
carried by a single person. 

Although the present invention has been described with 
reference to preferred embodiments, workers skilled in the art will recognize 
15 that changes may be made in form and detail without departing from the 
spirit and scope of the invention. 
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WHAT IS CLAIMED IS: 

1. A method for the collection of visual and spatial position 
information from a moving platform, the method comprising: 

obtaining spatial position data relating to the spatial position . 
5 of the moving platform; 

obtaining video images from a plurality of video sources 

located on the moving platform; 
recording selected video images obtained from the video 

sources, wherein the order in which the selected video 
10 images are recorded is dynamically configurable; and 

correlating each recorded video image with the spatial 

position of the moving platform at the time that the 

video image was obtained. 

2. The method of claim 1 wherein the selected video images are 
15 recorded on a single storage medium. 

3. The method of claim 1 wherein the selected video images are 
recorded on a plurality of storage media. 

4. The method of claim 1 wherein the selected video images are 
recorded in a known sequential cycle. 

20 5. The method of claim 1 wherein the spatial position data 

includes data obtained from a receiver located on the moving platform and 
capable of receiving data from Global Positioning System satellites. 

6. The method of claim 5 wherein the spatial position data 
further includes relative spatial position data obtained from an inertial 

25 navigation system located on the moving platform. 

7. The method of claim 6 wherein the inertial navigation system 
provides six-dimensional relative spatial position data defined in terms of 
latitude, longitude, elevation, pitch, roll and yaw. 
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8. The method of claim 7 and further comprising correlating the 
relative spatial position data with the spatial position data obtained from the 
receiver. 

9. The method of claim 8 and further comprising adjusting the 
5 relative spatial position data to remove any errors contained therein when 

the relative spatial position data differs from the spatial position data 
obtained from the receiver. 

10. The method of claim 9 wherein the receiver provides sbc- 
dimensional spatial position data defined in terms of latitude, longitude, 

10 elevation, pitch, roll and yaw. 

11. The method of claim 1 wherein the spatial position of the 
moving platform is calculated from the spatial position data. 

12. The method of claim 1 1 wherein a time code is associated with 
15 each recorded video image and wherein the time code is recorded along with 

the spatial position of the moving platform at the time that the recorded 
video image was obtained. 

13. - The method of claim 1 wherein the video sources include a 
plurality of video cameras and a four-view generator. 

20 14. A method for determining the spatial position of an object 

from visual and spatial position information collected from a moving 
platform, the method comprising: 

obtaining video images in which the object can be seen from 
a plurality of video sources located on the moving 
25 platform; 

recording at least two of the video images obtained from the 
video sources; 

obtaining spatial position data having six degrees of freedom 
relating to the spatial position of the moving platform; 
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determining for each recorded video image the spatial 
position of the video source from which the video 
image was obtained at the time that the video image 
was obtained; and 

5 determining the spatial position of the object from the 

recorded video images and the determined spatial 
positions of the video sources by photogrammetry. 

15 - The method of claim 14 wherein the recorded video images 
used to determine the spatial position of the object are obtained from njore 

10 than one video source. 

16 - The method of claim 15 wherein the recorded video images 
used to calculate the spatial position of the object are obtained at different 
times. 

17- The method of claim 15 wherein the recorded video images 
15 used to calculate the spatial position of the object are obtained at the same 

time. 

IK - The method of claim 15 wherein the video sources of the 

recorded video images used to determine the spatial position of the object 
are not co-planar. 

20 !9. The method of claim 14 wherein the recorded video images 

used to determine the spatial position of the object are obtained from a 
single video source. 

2() - The method of claim 14 wherein the order in which the video 

images are recorded is dynamically configurable. 

21 • The method of claim 14 wherein the spatial position of the 

video source from which each recorded video image was obtained at the 
time the video image was obtained is calculated from the spatial position 
data. 



25 
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22. The method of claim 14 wherein the spatial position data 
includes data obtained from a receiver capable of receiving data from 
Global Positioning System satellites. 

23. The method of claim 22 wherein the spatial position data 
5 further includes relative spatial position data from an inertial navigation 

system located on the moving platform. 

24. The method of claim 23 wherein the inertial navigation system 
provides six-dimensional relative spatial position data defined in terms of 
latitude, longitude, elevation, pitch, roll and yaw. 

10 25. The method of claim 24 and further comprising correlating the 

relative spatial position data with the spatial data position obtained from the 
receiver. 

26. The method of claim 14 wherein a time code is associated with 
each recorded video image and wherein the time code is recorded along with 

15 the spatial position data obtained at the time that the recorded video image 
was obtained. 

27. A method for the collection of visual and spatial position 
information from a moving platform, the method comprising: 

obtaining spatial position data from a receiver capable of 
20 receiving data from Global Positioning System 

satellites; 

obtaining relative spatial position data from an inertial 
navigation system located on the moving platform; 

obtaining video images from a plurality of video sources 
25 located on the moving platform; 

recording selected video images obtained from the video 
sources; 
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correlating the relative spatial position data obtained from the 
inertial navigation system with the spatial position data 
obtained from the receiver; and 
correlating each recorded video image with the spatial 
5 position of the moving platform at the time that the 

video image was obtained. 
28. The method of claim 27 wherein any errors in the relative 

spatial position data obtained from the inertial navigation system are 
corrected using the spatial position data obtained from the receiver. 
10 29. The method of claim 28 wherein the relative spatial position 

data from the inertial navigation system is obtained with greater frequency 
than the spatial position data from the receiver. 

30. The method of claim 27 wherein the inertial navigation system 
provides six-dimensional relative spatial position data defined in terms of 

15 latitude, longitude, elevation, pitch, roll and yaw. 

31. The method of claim 30 wherein the receiver provides six- 
dimensional spatial position data defined in terms of latitude, longitude, 
elevation, pitch, roll and yaw. 

32. The method of claim 27 wherein the spatial position of the 
20 moving platform is calculated from the spatial position data obtained from 

the receiver and the relative spatial position data obtained from the inertial 
navigation system; 

33. The method of claim 27 wherein the order in which the 
selected video images are recorded is dynamically configurable. 

25 34. The method of claim 33 wherein the selected video images are 

recorded on a single storage medium. 

35. The method of claim 33 wherein the selected video images are 

recorded in a known sequential cycle. 
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36. The method of claim 27 wherein a time code is associated with 

each recorded video image and wherein the time code is recorded along with 
the spatial position of the moving platform at the time that the recorded 
video image was obtained. 
5 37. A method of updating a Geographic Information System 

database using visual and spatial position information collected from a 
moving platform, the method comprising: 

obtaining a Geographic Information System database, wherein 
the database includes spatial position data; 
10 obtaining video images from a plurality of video sources 

located on the moving platform; 
obtaining spatial position data relating to the spatial position 

of the moving platform; 
correlating each video image obtained with the spatial 
15 position of the moving platform at the time that the 

video image was obtained; and 
replacing selected data in the database based on the video 
images and spatial position data obtained. 

38. The method of claim 37 wherein the Geographic Information 
20 System is a United States Census Bureau TIGER/Line file. 

39. The method of claim 37 wherein the database includes data 
relating to the spatial position of the beginning and end of a plurality of 
street segments. 

40. The method of claim 39 wherein replacing selected data 
25 includes replacing data relating to the beginning and end of selected street 

segments. 

41. The method of claim 40 wherein the database further includes 
data relating to the shape of a plurality of street segments and wherein 
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replacing selected data includes replacing data relating to the shape of 
selected street segments. 

42. The method of claim 37 wherein the spatial position data 
includes data obtained from a receiver capable of receiving information from 

5 Global Positioning System satellites. 

43. The method of claim 42 wherein the spatial position data 
further includes relative spatial position data obtained from an menial 
navigation system located on the moving platform. 

44. The method of claim 37 wherein selected video images ' 
10 obtained from the video sources are recorded and wherein the order in 

which the selected video images are recorded is dynamically configurable. 

45. An apparatus for the collection of visual and spatial position 
information from a moving platform, the apparatus comprising: 

means located on the moving platform for obtaining spatial 
15 position data relating to the spatial position of the 

moving platform; 
a plurality of video sources located on the moving platform 

for obtaining video images: 
recording means for recording selected video images obtained 
20 from the video sources, wherein the order in which the 

selected video images are recorded is dynamically 
configurable; and 
correlating means for correlating each recorded video image 
with the spatial position of the moving platform at the 
25 time that the video image was obtained. 

46. The apparatus of claim 45 wherein the selected video images 
are recorded on a single storage medium. 

47. The apparatus of claim 45 wherein the selected video images 
are recorded on a plurality of storage media. 
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48. The apparatus of claim 45 wherein the selected video images 
are recorded in a known sequential cycle. 

49. The apparatus of claim 45 wherein the means for obtaining 
spatial position data includes a receiver capable of receiving data from 

5 Global Positioning System satellites. 

50. The apparatus of claim 49 wherein the means for obtaining 
spatial position data further includes an inertial navigation system. 

51. The method of claim 50 wherein the inertial navigation system 
provides six-dimensional relative spatial position data defined in terms of 

10 latitude, longitude, elevation, pitch, roll and yaw. 

52. The apparatus of claim 45 wherein a time code is associated 
with each recorded video image and wherein the time code is recorded along 
with the spatial position of the moving platform at the time that the 
recorded video image was obtained. 

15 53. The apparatus of claim 45 wherein the video sources include 

a plurality of video cameras. 

54. The apparatus of claim 53 and a digital matrix router for 

routing video signals from the video sources to the recording means in a 
desired order. 

20 55. The apparatus of claim 45 wherein the recording means 

comprises a video tape recorder. 
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